When video is streamed over the Internet and played back through a Web browser or media player, the video is delivered in digital form. Digital video is also used when video is delivered through many broadcast services, satellite services and cable television services. Real-time videoconferencing typically uses digital video, and digital video is used during video capture with most smartphones, Web cameras and other video capture devices.
Digital video can consume an extremely high amount of bits. Engineers use compression (also called source coding or source encoding) to reduce the bitrate of digital video. Compression decreases the cost of storing and transmitting video information by converting the information into a lower bitrate form. Decompression (also called decoding) reconstructs a version of the original information from the compressed form. A “codec” is an encoder/decoder system.
Over the last two decades, various video codec standards have been adopted, including the H.261, H.262 (MPEG-2 or ISO/IEC 13818-2), H.263 and H.264 (AVC or ISO/IEC 14496-10) standards and the MPEG-1 (ISO/IEC 11172-2), MPEG-4 Visual (ISO/IEC 14496-2) and SMPTE 421M standards. In particular, decoding according to the H.264 standard is widely used in game consoles and media players to play back encoded video. H.264 decoding is also widely used in set-top boxes, personal computers, smart phones and other mobile computing devices for playback of encoded video streamed over the Internet or other networks. A video codec standard typically defines options for the syntax of an encoded video bitstream, detailing parameters in the bitstream when particular features are used in encoding and decoding. In many cases, a video codec standard also provides details about the decoding operations a decoder should perform to achieve correct results in decoding. Often, however, the low-level details of the operations are not specified, or the decoder is able to vary certain implementation details such as memory utilization to improve performance, so long as the correct decoding results are still achieved.
Video decoding can be memory intensive. During decoding, some buffers store encoded video data. Other buffers store various types of side information used during decoding for pictures of a video sequence, and still other buffers store reconstructed versions of pictures, each of which can include millions of sample values. Further, in some application scenarios, a device concurrently plays back multiple video clips. This might occur, for example, when a Web page includes multiple video tags for video clips that are played back at the same time. Or, it might occur when a video editing tool opens multiple video clips for editing or playback. For such application scenarios, video decoding can be especially memory intensive.